Protobuf Config #263
Merged
Protobuf Config #263
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This approach reduces the binary size by ~200kb
nanopb would not generate code section with enum type defines if the source file does not contain any enums itself. However enums can be included from other source files.
Implemented toJson() and fromJSON()
No more loading / saving of ConfigLegacy structs
It is not part of the 0.7.1 release
Typo? |
Must have gotten lost in a merge
Yes, fixed. Thanks. |
bsstephan
reviewed
May 19, 2023
arntsonl
previously approved these changes
May 25, 2023
| @@ -0,0 +1,52 @@ | |||
| function (compile_proto) | |||
| find_package(Python3 REQUIRED COMPONENTS Interpreter) | |||
There was a problem hiding this comment.
We'll want to make a note that we require Python again for protobuf since we removed that previously.
| @@ -20,8 +22,16 @@ class PS4ModeAddon : public GPAddon { | |||
| virtual std::string name() { return PS4ModeName; } | |||
| private: | |||
| struct mbedtls_rsa_context rsa_context; | |||
| uint8_t hashed_nonce[32]; | |||
| PS4Options ps4Options; | |||
| mbedtls_mpi_uint bytesN[64] = {}; | |||
There was a problem hiding this comment.
Was removing PS4Options for something with zeroing out the structs? All good either way!
| bool displayInvert; | ||
| int displaySaverTimeout; | ||
| ButtonLayoutCustomOptions buttonLayoutCustomOptions; | ||
| char boardVersion[32]; // 32-char limit to board name |
There was a problem hiding this comment.
We might be able to use boardVersion when doing legacy conversions
|
Bug #1: Custom LED enable mode does not save
|
arntsonl
added a commit
that referenced
this pull request
Jun 12, 2023
* Add an option to allow switching Touchpad and Share buttons (#254) Add an option to switch between Touchpad and Share buttons for PS4 Mode * Update development.md (#261) typo embeded to embedded * Update usedPins on save on /pin-mapping page (#262) * Update button labels for button label change and TP-Switch swap (#266) * Fix serialization of switchTpShareForDs4 * Add button labels for PS4 * Setup button labels for PS4 with TP & Share switch * Remove outdated Buttons constants * Update references to GP2040 in www (#264) * Update issue templates * SNES Controller Input Support (#252) * SNES controller support * adds webconfiguration options for SNESpad controller input * fix: corrects NES button mapping * fix[SNESpad]: updates lib to latest with snes mouse to analog working correctly * Fix invalid reference to button label (#279) * Protobuf Config (#263) * Build makefsdata for Linux w/ MUSL * Restored deflate support to makefsdata * Added nanopb library * Initial version of Proto config * Added AnimationOptions_Proto * Storage::toJSON() WIP * Pretty JSON output * Use std::string instead of std::ostringstream This approach reduces the binary size by ~200kb * Simplify ToJSON() and implented repeated support * Default initialization * Fixed issue in nanopb_generator.py nanopb would not generate code section with enum type defines if the source file does not contain any enums itself. However enums can be included from other source files. * Moved Config functions to separate file Implemented toJson() and fromJSON() * Restored StorageManager to its pristine state * Config loading and saving * Make FlashPROM cache public for the time being * Implemented api/getConfig and api/setConfig endpoints * Moved most Gamepad enums to enums.proto * Use DEFAULT_INPUT_MODE and DEFAULT_DPAD_MODE * Renamed generic/short enum values * Moved bulk of config structs to the ConfigLegacy namespace No more loading / saving of ConfigLegacy structs * Moved GamepadOptions and GamepadHotkeyEntry to ConfigLegacy * FlashPROM::start() is no longer needed * Added more settings to config.proto * Support for the latest config changes from main * Store config struct on the heap to avoid stack overflow * Increase LWIP_HTTPD_POST_MAX_PAYLOAD_LEN to 8kb We need more to fit the whole config * Further stack optimizations Only save config if content has changed * Removed template to reduce code bloat * Update config.boardVersion and save after ConfigUtils::load() This makes sure that any migrations performed are preserved in EEPROM * Config instance in StorageManager * Renamed getAddonOptions() -> getLegacyAddonOptions() Renamed setAddonOptions() -> setLegacyAddonOptions() * Added getters for sub-configs * Migrate PinMappings to Protobuf * Fixed wrong name caused by overeager renaming * Migrated reverse addon to Protobuf storage * Mirrored latest version of AnimationOptions in Protobuf * Fixed compatibility with arm-none-eabi-gcc 11.3.1 * Fixed crash issue in Release builds * Fixed board configs * Added commented out PICO_DEOPTIMIZED_DEBUG to CMakeList.txt Use this to toggle between optimized / unoptimized builds * Added support for byte arrays to base64 encoder / decoder * Store binary data as byte arrays, convert to Base64 in JSON * Migrated api/getSplashImage and api/setSplashImage * Change footer magic to invalidate incompatible stored data * Refactor pin assignment from JSON * Add bool to int workaround for web-config * Migrate SOCD Slider addon to ProtoBuf * Migrate On-Board LED addon to ProtoBuf * Migrate Analog Input addon to ProtoBuf * Migrate BOOTSEL button addon to ProtoBuf * Migrate Buzzer Speaker addon to ProtoBuf * Migrate Dual Directional Input addon to ProtoBuf * Migrate Extra Button addon to ProtoBuf * Migrate I2C Analog ADS1219 addon to ProtoBuf * Migrate Joystick Selection Slider addon to ProtoBuf * Migrate Player Number addon to ProtoBuf * Migrate Turbo addon to ProtoBuf * Migrate Wii Extension addon to ProtoBuf * Migrate PS4 Mode addon to ProtoBuf * Check writeCache when determining whether to save Due to the delay in FlashPROM we run the risk of delaying the save indefinitely * We only allow saves from core0. Saves from core1 have to be marshalled to core0. * Migrate AnimationOptions * Check serial and signature for size as well * Removed unncessary cast * Moved PS4Options to config_legacy.cpp * Migrate Display addon to ProtoBuf * Removed old code related to SplashImage * Removed old code related to AddonOptions * Removed stale functions from ConfigManager * Moved OnBoardLedMode to config_legacy.cpp * Fixed issues caused by the merge * Added new player LED properties to LEDOptions * No more usage of ConfigLegacy::GamepadOptions * Added switchTpShareForDs4 to config.proto * Conversion from legacy storage to Protobuf * Return -1 for invalid pins from webconfig * More conversion from legacy storage to Protobuf * Validate enums when converting from legacy config * Remove switchTpShareForDs4 from legacy conversion It is not part of the 0.7.1 release * More conversion from legacy storage to Protobuf * Removed references to ConfigLegacy::BoardOptions * Fixed compilation of KB2040 * Re-added /api/getConfig function entry Must have gotten lost in a merge * Fixed issue where JSLider addon fails to change dpadMode * Migrated SNES options to Protobuf * Fixed saving of AnimationOptions --------- Co-authored-by: deeebug <[email protected]> * Make danger zone an actual button (#284) * Reference splash bitmap data from BoardConfig.h (#282) * Reference splash bitmap data from BoardConfig.h * Remove unused constant for bitmap * Add documentation on default splash customization * .gitignore a couple files used by vim + tools (#267) * Implement Forced Setup Mode & Hotkey Lock settings (#273) * Implement Forced Setup Mode & Hotkey lock settings * Rename enums * Add warning to web-config for Forced Setup Mode * Update warning check text * Fix BoardConfig.h * Feature: webui color schemes (#295) * WIP * DarkMode is a Component not Service * Give the theme button a right margin * Use Dropdown from `react-bootstrap` and dynamically create dropdown items * Rename to ColorScheme * move colorScheme to AppContext * Stop setting localStorage here * Simplify setTheme logic and ensure data-bs-theme is 'light' not 'auto' when light system scheme * Remove unused imports * More contrast for the navbar * Fixed PS4Options saving (#298) Fixed ps4Options saving * Move invert Y to process() to avoid hotkey issues (#300) * Preparing 0.7.2 for release candidate (#310) * Fix #306 (#307) * Fixed PLEDs not being properly retrieved from protobuf (#311) * Fix #265 (#312) Fix external links * Disabled /api/getConfig (#318) * Add Stress FightPad (#319) * Add Stress FightPad * Add controller photo * New photo, blurred the NSFW text * Updating config for full censor, docs/index.html will be updated when release goes out. * Exclude fields from JSON config export (#322) * Fix button labels and hotkey when Touchpad Share are swapped (#317) * Fix Hotkey when Touchpad and Share are swapped in PS4 Mode When the Touchpad and Share buttons are swapped in PS4 Mode, Hotkey F1 will be the Touchpad button, and Start will not be the Option button, but the Start button. * Fix button label when touchpad and share swapped in ps4 mode When selecting button label is not the PS4 button label, the button label may be different. * Fix button label when touchpad and share swapped in ps4 mode When selecting button label is not the PS4 button label, the button label may be different. * Fix button label when touchpad and share swapped in ps4 mode When selecting button label is not the PS4 button label, the button label may be different. * Fix button label when touchpad and share swapped in ps4 mode When selecting button label is not the PS4 button label, the button label may be different. * Fix WebConfig button label when touchpad and share swapped in ps4 mode When selecting button label is not the PS4 button label, the button label may be different. * This change did not make sense to be part of the web config modification. * Doc updates for v0.7.2 (#323) * Doc updates * Remove unused image * Styling updates --------- Co-authored-by: deeebug <[email protected]> Co-authored-by: Wren <[email protected]> Co-authored-by: Pelsin <[email protected]> Co-authored-by: Henré Botha <[email protected]> Co-authored-by: TheTrain <[email protected]> Co-authored-by: Robert Dale Smith <[email protected]> Co-authored-by: Malte Thiesen <[email protected]> Co-authored-by: Oliver Sayers <[email protected]> Co-authored-by: Brian S. Stephan <[email protected]> Co-authored-by: FeralAI <[email protected]> Co-authored-by: teikai1216 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the promised Protobuf refactoring. This PR contains A LOT of changes:
proto/config.protoandproto/enums.proto..protofile actually changed. The only requirement is a Python3 installation.api/getConfigandapi/setConfigendpoints are implemented in Webconfig to read / write the whole config at once. They use the JSON functionality and will also adjust to schema changes.api/setConfigwill correctly return 400 if it gets passed invalid JSON and 500 if saving fails for whatever reason (it should not).BoardConfiginto multiplestructs as well asAddonConfig. I used the structure of Webconfig UI and the Webconfig API as a rough guideline.The following was tested by me:
The following addons should still be tested by those with the required hardware:
I'd also like to hear feedback for the current config schema. If we want to change something it will be much easier to do this now than later.
I will provide proper documentation that explains how to change the config schema with an additional PR.